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ABSTRACT 

Studies of methods of teaching statistics are 
repbrt^d. Three brief texts were written\for the investigations; they 
varied in degree of explanation of basic concepts of elementary 
probability. Subjects were undergraduate students at the University 
of Massachusetts, Amherst. None of the subjects had had previous 
formal exposure to probability or statistics. All texts presented. six 
basic formulas from elementary probability and an example of the 
application of each formula. All three' documents contained 
definitions of special terms. The high-explanatory text emphasized 
the relation of probability to counting, used- pictorial aids, and 
presented equatipns as natural developments ensuing^ f rom examples. 
The standard text lacked this conceptual development, but presented 
more concrete examples than those found in the low-explanatory 
document. Subjects were given sufficient time to read their text, 
t.hen given an unrelated 15-minute task to minimize immediate memory 
dependence. This ^as followed first by a performance test, th^n,by an 
aptitude measure,. Result g] suggested that very different patterns ' 
appear. to be present, in tne "high-explanatory subjects as compared to 
the other groups. In particular, subjects in the high-ex!^^a^tor^ 
group tended aot to retrieve formulas, but to recall examples. 
(MP) / 
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Elementary probability provicJtlk the foundation on wh^ch statistical 
theory and practice is built. It^fcpportance as a discipline goes well 
beyond that, however. Frequently, |&e average citizen encounters probability 
concepts and statistics *in his jreading and in his viewing of television 
newscasts. To cite but a few examples, we are all exposed to information 
about statistical test results and correlation in discussions of the 
role of smoking in cancer, to information about confidence intervals 
^ in presentations of results of polls prior to elections, and even to 
probabilities in viewing sports and the weather. Many of these sorts 
of "everyday examples" are discussed in an excellent book of readings 
edited by Tanur (1973). We contend, then that statistical education 
is an important part of the general education of students, and therefore 
it is critical* to attend closely to the fundamental concepts and calculations 
to be found in the area of elementary probability. 

Despite the clear importance of the topic, there is no evidence that 
it is being taught properly. Indeed, research in th£ la£t decade (Tversky 
and Kahneman, 1974, provide a seminal paper) has made clear that college 
students have marked misconceptions about probability and statistical 
concepts, kg Tversky and Kahneman note: "Misconceptions of chance are 
not limited to naive subjects. 11 They go on to report results of studies 
conductd w£th research psychologists, individuals who ordinarily have 
had several statistics^ courses at the undergraduate and graduate level. 

It appears time that systematic investigations of methods of teaching 
statistics were undertaken. The research conducted under the auspices 
of this grant provides a beginning. We begin with the observation that 
most introductory statistics texts emphasize memorization of formulas 
with little conceptual development. Recently, there have been a few- 
exceptions to the general approach; Freqdman, Pisani, and Purves (197§) 
and Pagano (1981) are examples of an approach that places a greater^-emphasis 
on understanding. We have conducted several studies contrasting these 
approaches and have obtained some interesting results. In what follows, * 
I will summarize the general prooedu^res and results, and provide a brief 
discussion. Three appendices to this report provide a considerably greater 
- detail. The paper by Myers, Hansen, Robson,*and McCann (Appendix 1) 
has been accepted for .publication, that by Hansen, McCann, and Myers , 
(Appendix 2) has been submitted for publication, and that by Myers (Appendix 
3) constitutes an invited presentation to the First International Conference 
on tile Teaching of Statistics and will appear 'in the published proceedings 
of that conference. 

Methods 

Subjects 

Subjects in these studies were all undergraduate students at the 
University of Massachusetts, Amherst. None had previous formal exposure 
to probability or statistics. 

Instructional Texts - K 

Three brief texts were written for the purpose of this research program. 
During the course of the research program, r evis ions were developed to 
Jfaprove clarity of presentation. Such revisions were based on taped 
interviews with subjects who commented aloud as they read the text. 
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All three texts presented six basic formulas from elementary probability 
and an example of the application of each formula. All three texts contained 
definitions pf such terms as independen9e and mutual exclusivity. In 
t>he high-explanatory iext, the relation of probability to counting was 
emphasized, pictorial/ aids were used, and the equation was presented 
as a natural development ensuing from the example. The standard text 
lacked this conceptual development but presented examples that were some- ' 
what more concrete than those employed in a low explanatory text. Further 
detail on the nature of the texts can be found in the Appendices. 

Performance tests 5 

In most of our studies these fests involved equal numbers of formula 
(e.g. "If P(A) = .4 and P(B) = .6, what is P(A and B)?") and story problems. 
There was one study (Appendix 2) in which only story problems were presented. 
Where both types of problems were presented, the order of presentation 
was counterbalanced. Examples of problems can be found in Appendix 2. 

Procedure 

Subjects were given sufficient time to read their text and then we.re 
given an interpolated unrelated task for 15 minutes* to minimize dependence 
on immediate memory. They were then given sufficient time to do the 
performance test. . Following this, they were given an aptitutde t$st 
based on items from the quantitative and analytical Graduate Record Examina- 
tions. 

Results and Discussion 

Very different patterns of knowledge appear to be present in subjects 
in\the nonexplanatory (standard and low-explanatory texts) and explanatory 
conditions. Subjects in the two nonexplanatory groups performed consiaarably 
less^pll on story than on formula problems^ and often used the correct 
'formuM for a problem (that is, met the lenient criterion) but failed 
tov solye it. A closer look at answer^ to story problems revealed that 
subjects in the nonexplanatory conditions often required the explicit 
presence of key words which unambiguously pointed to certain operations, 
tended -to miscjassify problems in the presence of irrelevant or redundant 
information, and made many errors when the values in the story required 
modification before insertion into the formula. In contrast, subjects 
in the high-explanatory condition performed equally well on story and 
formula problems, tended to solve whenever they showed evidence of knowing 
the appropriate formula, and. were considerably less hindered by absence 
of key words, the presence of irrelevant information, and the need to 
translate values in the* story. The situation may be summarized by noting 
that* although all- our sv^bjects were novices, those in the high-explanatory 
condition seem to share more of the characterised of experts tfran do 
those in the other ' two 'conditions . * - < 

Th£ results of more recent studies lead us to believe that the solution 
processes in the explanatory and Qonexplanatory^conditons are qualitatively 
different. In one study, subjects in a high-explanatory condition had 
only 60% correct .recall pf equations (as opposed to 90% for a standard 
group), but yrere able to perform correctly on 53% of story problesm. 
This result suggests to us (as does the equivalence of, formula and story 



scoreg.iin the study just detailed) that subjects in -the high-explanatory 
condition often may not retrieve the formula at all* Another study in 
which ^^ubjects thought aloud while solving supports this conjecture; 
after. studying a high-explanatory text, students tended to attempt to 
con5t;E.¥£t solutions, often using the diagrammatic aids provided by the 
text, feather than recalling formulas, they .attempted to recall examples 
from t^e text which they felt were similar to the problem at hand, and 
to us^^these 2s a model for solution . 

Much remains to be done. JHigh on the agenda must be the use of other 
measujf£.s of comprehension^-both problems demanding more complex processing 
and gr^nsfer to new materials such as the Binomial and Bayes 1 theorems ♦ 
We suspect that such research will provide further evidence of the' role 
understanding can play in mathematical problem-solving. 
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Abstract 



(In \his study, 48 subjects who had no previous exposure to probability or 

Statistics read one of throe texts which varied in the degree of explanation 

of> basic concepts of elementary probability. All'texts contained six formulas, 

each accompanied by an example, as well as definitions and information 

logically required to solve all problems. The high-ekplanatory text differed 

from the low-explanatory and standard texts in that it emphasized the logical 

a* 

basis underlying the construction of the formulas, the relations among 
formulas, and t]^e gelations of variables to .real world objects and events. 
On both immediate and delayed performance tests, subjects In the low-explanatory 
and standard text conditions performed better on formula than on story 
problems, while the subjects in the high-explanatory text condition did 
equally well on both types of problems. ^It was concluded that explanation 
did not improve the learniifg of formulas but rather facilitated the applica- 



j . 



tion of what was learned to story problems. 
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Parents, educators, and mathematicians have been involved in recurrent 

debates about the relative merits, of understanding mathematical concepts 

and rote memorising computational formulas. In pi^nciple, such conflicts 

should be resolvable through research involving manipulation of the degree 

to which students understand mathematical material and observation of their 

« • 
performance on tests of mathematical knowledge. Ln actuality, we enc9unter 

* f * 

difficulties because we lack both a precise definition of mathematical 
understanding and a ^specif ieation of the processes *uch understanding might 
affect. 

The present study examines the effects of varying understanding of 
elementary probability.. We have identified three dimensions of understanding 
and have constructed three texts that var**s^ong these di^ensijns. The 
first dimension is similar to what Mayer and Creeno (L972) have labelled 
"external connectedness. 11 Ln a high-explanatory text, probability was 
defined as a relative frequency of, events, and examples emphasized ' the 
relation between answer^ and counts of the numbers of ways tha4: various 
events could occur. A standard 1 text treated probability as a measure 
w^.th uertain well-defined properties (e.^;. limits of one and zero) and 
ust?tLcamiliar real-world examples *(e . g . the probability of rain tomorrow), 
but did not cxplairj probability in terms of counting processes. A low- 
expLanatory text was more abstract than cither of the fir.;t two; aside 
from an initial reference lo dice-throwing, examples were stated In terms ' 
of such quantities as P(A) and P(B). 

The seconded Intension along which the texts .varied wis the degree of 
linkage among, ^he equations presented. Only the high-explan'atptfy text 
shoWfed that one equation was a special case of another, or that one equation 
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could be derived from another. For example, this text pointed out that the 
probability of the union of two mutually exclusive events was a special 
case of the probability of the union of any two events. 

The third dimension af relevance was the degree of explanation provided 
for each equation. In the high-explanatory text, Venn and branch diagrams* 
were used to represent the information in the examples, and the equations 
applied in theSe examples were explained in terms of these diagrams. Neither 
of the other texts provided such explanations. For example, although all 
three texts stated that the probability of the joint occurrence of two 
independent events was the product of tlleir probabilities and provided 
an example of this, only the lii^-explapatory text attempted to demonstrate 
why probabilities were multiplied in such cases. 

Learning was measured by performance on two^types of problems: f ormul a 

problems , which merely presorted certain values fo .ft. f T(\) = v .6, P(B) = .4") 

and necessary conditions ( M A and B are independent events ") , -4ind required 

the subject to set up the correct results (e.^. "What is iHA^and B)? 11 requires 

.6 x .4 as the answer); and story problems which were considerably less 

transparent. Solving formula problems should depend only Vp\»n memory of 

the equations, while solving story problems should depend aljs'o upon the 

* ) * 

ability to apply the eauations; that is^tp select the oqiutiou appropriate 

© • ■ 

to the problem and then to insert the appropriate number? from the story 
into t.4ie selected equation. 

It is not clear which y»roup should better remember t lie equations. 
There is some evidence that spending time and effort learning added, 
elaborative, material may hinder the acquisition of certain basic points 
(Reder and Anderson, 1980). .In a like manner, the high-explanatory^group 
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may spend proportionately less time and efflrt than the standard and low- ' 
explanatory groups on learning the equations'; the high explanatory text 
involves 14 pages of materia! compared to four pages for Lfie. other two 
texts. On the other hand, subjects in thp high-explanatory group should 
be able to relate each equation to bp'th their world knowledge and- the other 
equations; the required equations would therefore be more likely to be 
stared in memory, or stored with greater strength in memory. Such 1 
consequences of integration have been demonstrated by many investigators 
(e.g., Bransford & Johnson, 1972; Kintrxh 6 van Dijk, 1978). Finally, the 
formulas may be easier to retrieve- in the high-explanatory, condition. _ 
When concepts are linked to many other concepts, there is a greater 
likelihood that the retrieval of any one piece of informal ton will auto- 
matically ,! prime M the retrieval of other Infold:; Lion (Ct ! I Lr . $ Loftus,. 
1975). 

a 

These arguments Indicate liiat the* l^igh-exp laaa tory group has 
potential advdntages and disadvantages in learning and retailing the for- 
mulas. Thus, no text condition has an indisputable advantage on formula 
problems. Nor is it clear which group, if any, should perform best on story 
problems. Such problems may be viewed as involving three, stages : cate- 
gorization of the problem,'* retrieval of the formula appropriate for that ; "" 
category, and translation - correct substitution of values from die story 
■ into the retrieved equation. * We would expect that because the high- 
explanatory text places an emphasis on understanding concepts and their' 
connections to real-world referents, subjects in this condition "would have 
an advantage in categorizing and translating- story problems. As we noted 
above, however, the o.ther texts may have an advantage in learning the 
formulas and. therefore, in retrieving them. * ' 
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Although the potential advantage of the Low-explanatory and standard 

groups i^^etrieving the formula makes it difficult to predict the outcome 

of comparisons between groups, there are two predictions that fallow from 

\ 

the stage analysis just presented. First, because story problems involve 
categorization and translation, in addition to retrieval, they should prove 
considerably more difficult than formula problems; Second, this advantage 
of formula problems over story problems should be much less for the 'high- . 
explanatory group than for the other two groupb. In the high-explanatory t< 
external connectedness should aid^in translating the comoonents of a story 
problem; and linkage among equation's and an understanding Lostered by 



\ 
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the' explanation of each equation nay aid the subject in n -o:-.. trading the 
required equation. 

V^Thrs ^re'JjjCtio.ji of- inter ic lion finds some support i.i a studv by Mayer 
I > 

(1974); he 'iuund that subjects h jc! more difficulty solving story problems 
than formula problems and the effecL wa^ greater i.n a te*L pnnha r .LZing 
calculations (formul s i text), Mayer taught the binomial ^iiuorein lising twe 
texts which differed in their emphases on cdlculat u ns aid links to experience 
and in the' sequent in e. of £q formation. However , his °,onvi \l and fcrnula 
texts are very different fr^n any of our Lnreo, te:.t \ in -atent, emphasis,, 
sequencing of in format ion , and 'ewj'th; and there niir^d di -ic^cnces 
in oflir procedures and test prob terns. One poionriailv i. ^-u ).'t difference 
is that Mayor' subject.*, had the binomial formula u> fivn oi thorn while 
taking the test so that the retrieval stage may not have ta critical, 
as in \ho present study. Thus v. diffVrent pattern of results is possible 
in' r' m \v. pLCsentt study . 

Parker (aote I) used materials more similu* to t husc used in^ the present 
study. Sht tested ci^hl subjects in each «>t two conditions, similar UQ 
our standard and higb-explanrtory conditions. She aJso I ouitd that story 
problems were more difficult than formula problems, and that this difference 
was more marked for her standard -*',roup. Following completion of that study, 
the hij;h-e>.r)lanat6ry te*t was codified to include questions dedtt&ncd to 
t*p the subject's understanding op the tqxt. 'Tlits ouobtion-and -answer 
version was. then read b) oif.ht n^w subjects, who choufcht aloud as they 
considered the? interspersed c;uestion.». Their comiaoaits were tape-recorded 
and analyzed. On t>.« basis of "ih.it analysis, several changes were made 
in the text used in this experiment. These changes were designed to clarify 

* ft * 
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the presentation. Subjects' responses on a questionnaire tfiven at the 
end of the present experiment indicated that this, vo.il was achieved. 

Tn the present .study, wo have twice as many i.ubjcr.Ls in each condition, 
have added the low-explanatory text condition, have counterbalanced the 
order of po^ttest* (which Parker tailed to do), and asked subjects to set 
up the answer rather inah to provide a final numerical result. This last 
procedural change allowed us to examine the possibility that subjects 
might use the correct formula but not substitute values correctly in it. 
This is important because our theoretical analysis suggests that such partially 
correct responses should be more prevalent in the* standard and low-expianatory 
groups, which, presumably, have more difficuity wiuh what we" have called 
the translation stage. 

N Metho d 

Sub'j ects , i 

Forty-eight volunteers fiom an Inl ro<hictor\ Ps>choloj;y course we're 
randomly assigned to three text conditions. None had previous exposure 
to probability or statistics. 

Text Materials % '* 

There wire three texts: l'ow-explanatory , standard, and high-explanatory 

All began with a brief introduction indicating that the text contained 

» • / 
a series of examples of probability calculations, and that each exam ple 

would be followed by a formula which could he used to solve; problems of 

that type. All three texts then contained a section label UtI "What are 

the chances?" Which introduced t'he concept of probability and its limiting 

values. This section noted th.it an action such ah tossing a coin or observing 

the d«iYv's weather could have several possible outcomes, liut a probability 
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Value could bo assigned to these outc*>mos, and* th.'t thvse values ranged from 
zero to- one. The low-explanatory te:ct tnen e.vpl lined tint the pqssible 
outcomes 'could be d*.s ignited by letters, vnd introduced terms such as P(A^ 
and P(B). The hif»h-explanjt'ory text ws* the only text to present probability 
as a relative frequency of outcome^. It presented a concrete example (a 
jfar with six red and four white marbles) and expl.iincd^ fiat the probability 
of drawing a particular color was the proportion of marble of that color. 
The remainder of the three te<ts presented six exruiples ^nd formulas. 



The formulas are presented in L'abl 
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The tiexts are too ien^thv u> 
difference in aoproachc^ may be ob 
in Table 1 . The low-explanatory t 
way : £ 

"Two events, A and B, dre s^i 
can ocrur on \ single trial 
liead or tail on a sin*;lc| toss 
If A and B are incompatible, 
P(A .or ft) — is the sum o£| the 



and P(B)' = .45/ Then, 
^V(\ or B) * P{A) + t 
- .20 + 



Wc miftht have several incompatible! 



reproduce hero but soir.e sense of the 

A* 

ained by con^ ide^inr. the. first formula 

xt presented this, mater ial in the following 

:1 to be in .c < gr p ; ud b 1 e if onlv one of them <^ 

- 

For example^a coin can come up either 
so that these two events are_n.ncompatib.le. 
the probability that A or B occurs — 
two probabilities. Suppose P(A) = .20 



.45 ■ 



E, etc". Farther suppose thntj wo y. 
anil P(C) , * Then, t 

. P(A or B or C) « P(A) + 



.65 



events which weMl call A, B, C, D, 
low the nur.ieric.il values o # f V(A) , P(B), 



,15 
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.Oat of a total of 100 marbles, we have 45 chances of .i ravin" * reel 

, plus 20 chances o) drawing a white one. Therefore, the prob ibility 

' \ 
of drawing a red or a white marble is 

45 + "'0 

P(red or wTiite) » — ^qq — 

= 45 , 20 
loo 100 

= P(red) + (white) 

We say that red and white nrf incompatible events bo;ause only one 

can occur on a single draw; the marble can be red, or white, but not 

both. A' picture r;ay help.'' 

1 At this" point, a I igure containing a rectangle with two nonovbrl apping 

circles was presented. One circje was labelled M 45 red" and the ochcr 

i 

■"20 white**; the area within the rectangle but outside the ircl^s was labelled 

"35 other." Following this, the extension to more events wab roted and 

♦ 

Equation l (wSte presented* 

Tn summary, all texts contained an example followed by the equation, 
as well as a sufficient definition of mutual, exclusivity. In the high- 
explanatory text, the relation of probability to counting was emphasized, 
pictorial 'aids we^e used, and the equation was pfl'SGiuq^ is a natural develop- 
ment ensuing from the example. , 



Perfornance T ests ^ t , . 

Two test:; were 1 constructed , each containing iwo setr of six story problems 
arid tJo sets of six formula problems. Within each set, there was one problem 

testing , each of the six equations in the text. The -order of sets within 

\ ' 

a tc^t wa:, counterbalanced over subjects &o that there were four .sequences,' 

s 

w\j^h the constraint that storv and formula setfe would alternate. Half of 
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the subjects received one tost fir\>t and the remaining subjects received 
the other test first. 



Formula and storv problems differed in the 'decree to ^jicli the <: 



iven 



information was explicit in "the question f For e::a-npJe, .1 formula problem 

testing knowledge of Equation 1 would slate !, P(A) - .4 and P(B) = .3. If - ' . 

A and B are incompatible, what is the .value of P(A or B)?" In contrast, 1 

a story * problem testing the same knowledge woyld read as follows: "The 

probability is .20 that Jiircny will win a race and. 30 that he will finish , 

second. What is the probability that be will* win or finish second?" ^ y 

Aptitude Tests .1 

A cen-ircm test measured knowledge cf basic algebra, decimal*, pcr- 
/ 

centals, fractions, and syrbol translation. A second, eifdtt-item test 
measured analytical ability; it required subjects to choo.se from among several • 
Venn diagrams the one that represented a specified relationship among real- 
world sets. The items in both tests were taken f ro,n the .iiulvtical abilities 

and mathematics sections of the Graduate Record Examination aptitude test. 

» 

^ Pro c edure 

J^ibjects wfire t.ested individually. They were instructed t«> rend the 
text booklets silently and at their own rate, trying to understand the explanations 

r 

and examples. The standard and low-explanatory groups, whose texts "were 

i 

four, pages in length"! were told that they would have 15 mi"iifes to finish; 



the high-explanatory group, whose text was l<* pa^'s i n length, was given 
25 minutes. These limits, based on pilot subjects, were i.iposod to discourage 
.any individual from either quuM> scanning the texts, or taking an inordinately 
long tiire* All subjects finished reading their tc:ts in the given time 
pei iod for. their fcroup. Subjects in the low-explanatory tnd standard groups 
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tended, to rate tho study time more favorably tluui subjects in the hiRh- 

2 

explanatory text condition, but this difference was not significant. 

At the completion of ^hc studv period, subjects were Riven twe minutes' 
-/ • 

to study a list of 15- words unrelated to probability and wer ( e then required 
to write down as many words as* they could recall on i blank sheet&of paper/ 
Upop completion of the interpolated task, subjects were gi^^e^/the test, 

m t 

,Tl)ere was' one .problem o^a page with space, to do any work which the subject 

felt mi'^ht help in the solution. Subjects w-rc «iven as r:nch cime as they 

required Lo^complVte t'iu test: most required abeut 30 nuruU'b. Upon completing 

the problems, subjects" wiire asked to return 13 hours 'l.it<r . the second 

test; day, subjects were Rivon the second/ ; : orm ox the test. Tho order of 

presentation of the tc^t forns was counter balanced. 

Following the second peri ormance- teit , subjects were eivun as much 
^ t 

time as they required to^ respond to the aptiLude tests and to a questionnaire, 

which asked subjects to. rate the study time, the cinaprcht i.^Lb ility of the 

text^ and the difficulty of the problems. Since tfiere were no significant 

differences between groups by any of these measures, the) will not be 1 considered 

further. / 

- t< 
/ Results • ** 

Performance test data were scored according to both a strict and ;i 

lenient t:r iterion. Tht? &Lrict criterion required chat Liu: subject sot up 

the answer correctly in all respects^ the Lenient criterion gavef credit 

if the subject used the correct equation but. for, example, inserted ihc 

wronq values. iThis distinction applies only to* pert'orninnt s on story problem?: 

strict and lenient scores were Dearly the same for formul.i problems, 
i 
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Performance Test .Data 

Xwo analyses of variance were performed on the data. The first analysis 
* » 
was oft the strict-criterion da Us. This involv :d three between-subject variables 

X three texts, four sequences of problems, and two orders oC post tests) and 

three withift-subject^ariahl es (two problem tvpes, si$" sets of problems 

with one correspond!^ to each of the six equations, and two days). The 

second analysis dropped one problem type, the formula problems, and introduced 

criterion (strict versus lenient) as variable. 

Figure 1 summarizes the results. Consider the analysis of the strict- 
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criterion daLa first. Formula problems had a Higher rate >1. solution than 
story problems when scores were averaged over text conditions and days; 
£(1,24) = 532.22, £ ^.OOl, MSE = ^6129. Problfcm type interacted with text; 
£(2,24) = 10.41, £ <.001, MSE * .6129. Two subsidiary analyses were performed 
to determine the source of this interaction. First, sinpic effects tests 
were performed comparing the three ti-xt conditions on formula and on story 
scores; despite the apparent superiority of the statical u and low-explanatory 
groups on formula problems, and of the hi);h-cxpljindLory .roup on story problems, 
' neither analysis yielded a significant result (p - .10). Analyses of covariance 
using aptitude scores as covViriates produced the same resirit. ' s 

In a second analysis, story and formula scores we're contrasted for^ 
each of the three text conditions. The low-explanatory ;ind standard groups 
did much better on formula than" on story problems while the hi^h-explanatory 
t;roup performed about- equal Iv well on both problem types. £ tests of prbblem 
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s 

: 

type were performed on the data for each group sepaialoi\: for Uio low- 

explanatory group, F(l,.!4) -"33.66, £ ' .001; for the si aiviard Rroup, 

£(1,24) - 39.16, j> '.001; t'or the high explanatory group . F < L . Because* 

the error variances were quite stable across the throe r.o.\t conditions', 

thfe pooled MSE of .6129 was the error term in all three :%*ts. 

The effect of days differed for the two types of problems, as indicated 

by the significant day \ problem type interaction; F(l ,2A) = 7.12,, MSE = ,1580, 
S 

^.02. There was a small nonsignificant drop in proportion < jrrect for 

* 

the formuLa problems from the firs,t to the second test (the proportions 

at>e^ .63 and .60, respectively), am! a small nonsignificant increase f6r 

the y tory problems 3 and .46)v 
c * " * * 

Table 2 presents the proportion correct for each grvup foa: each set 
of four problems testing knowledge of eaoh equation. Hie problem sets varied 



Insert Table 2 about here 



irt difficulty [F(5,120) = 21.72, MSE - 5.63, £ < .001] ; the degree of 
variability among the six sets depended upen .tuxt condition [j;(i0,120) = 2.09, 
MSU = .5163, £ < .05] and problem type [F(5,120) * 18.12, >[Sfi - .3566, 

2 < .001). Ono factor in this variability appear., to be that mean scores 

N 

for the standard and low-exp lan 1 tory groups are well below those of the 

<* 

high-e;-:p! aiXtifcory* group on all the story problem sets touting knowledge of 
Equations L, 2, 4, and 6 (Sut A) but not on those Lostinfl^iTnowlcdgc of Equatto: 

3 and 6 (Set B). In response to this observation, four sii^iif iaance tests 



were performed contrasting the high explanatory 



group against the other 



two groups combined on, Set A .story problems, on sVc_J^story problems; 
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on Set A formula prob^n^f and on Set formula probUms. Only the V rtftTo \ 

for Set A story problems wis greater than !; K(l,4>) « 6.10, £^ .025^^ 1 

. N This contrast is post hoc and ^he reported statistic siiould be viewed 

with caution; it would not be significant Ubim; ifehefte's criterion. \ Never- 

theless, the result makes sense in term? of the sta*;e model presented earlier. 

Story problems testing .knowledge of Equations 3 and 6 (Set B) contain ij 

key word, "and," wn£ch ' unambiguously dictates multiplication of the stated ^ 

probabilities. This is not true of other story problems; for example, "or" 

may dictate Equation L ( addition) or Lquation 2 (addition and subtraction)^ 

) * 1 

This difference between Set A and Set; 15 story problem^ suggests that the 

difference in rijiiildL in testing eontrasU* on the two sets reflects an inabili 

of subjects In non-xplanutory conditions Lj match the performap.ee of those 

in the hi£>h-e::p lavatory ycoup except when," cat egorid.it ion and translation 

requirements are minimal. ^ 

The analyses reported so far have been based upon a strict ^coring oritur 

% 

tp receive credit for ah answer, the subjects, had to have it entirely correct. 

Answers to story problems were also scored according to "a lenient criterion, 

which .accepted as correct the appropriate formula* regardless of whether 

values were correctly inserted. The criterion did not interact with the 

sequence of test problems tfihjtin or across days .(ail Fs •> 1) and, therefore, 

> 

these sources were pooled with the error term Cor, criterion and criterion 

by text condition to permit more powerful te:;ts. Lenient criterion scores 

were significantly higher than strict criterion score's; JlO>45) ^ 49. 5 ( *, 

» 

MSE = .3656, 2. ^ .01. The criterion interacted significantly with text 
condition, F(2,45) - 3.65. j> < -P5. The reason for this interaction appears 
to be that strict and lenient criterion scores differed much less in the 
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high-explanatory text condition than in the other two c onu • l lotis ; the inter- 
action sam of squares wa* complete! - ; accounted for by the v olv/rast, bef^een 
the high-exnlanJntory and till' other tvo conditions with ros'i *ct to the strict- 
lenient difference. The,F ratio for this contrast was 7 . 1 ( \ p < .05 using 
✓ - - 

Scheifc'ji . procedure. 

* a 

Aptitude Data * 

The respective mean aptitude scores for the low-explaiuiUvy » standard, 
and high-explanatory croups were the following (expressed m rroportion 
correct): Quantitative - .66. .66~, .7h; Analytical - .69, .71. .82. The 
differences were not 'significant?, both .£s > .10. L'arthennore, an additional 
analysis of performance scores for^X subset of .subjects matched in aptitude 
produced the sa^.e results as the original analysis reported in the preceding 
sections. ^ 

•~ Discussion 

I • ■ '■ — 

Consider three alternative models of tae role explanatory text. 

' An elabora tion model holds that the presence of additional, integrative 
and explanatory, material provides additional retrieval routes to the main 
points to be remembered (Reder & Anderson, 1980). That model provides an 
account q>f many memory ctanomuna, incJuding levels of processing results, 
and better memory for supurorcliiuuo ideas in passages (.Anderson, 1976). 
It would predict that, the high-explanatory group would have an advantage 
in retrieving formulas. It clearly does not provide an adequate account 
of the effects of our texts. Our high-explanatory croup ,; oes worse than 
the others (although not significantly) in solving formula problems. Further 

4 more,' in a study recently completed, we have found significant superiority 



of the standard group against Hie high-explanatory group (.90 vs. JbO correct 
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in recalling the n :ht hand *ide of the .equations to the cue c*f the *trn. 
hand side. 

While the eJ .unlr.iti.7n model focuso's on retrieval, the interfe rence uiodeT 
focuses an initia^ storage. It holds that the <idditional material, inmrcnt 
in an elaborated text takers needed resources and the main points are less 
well learned. P.euer and Anderson (1980) concluded that this is what happens 
'in studying chapters from texts in linguistics and geography. The resui ts 
on formula problems and memory for formula", nuied above, sugpe.?t that there 
nay be some truth to this. Formulas do appear to be loss well stored ia 
the hi^h-expiaoatory condition. Nevertheless, this model Ls "incomplete 
as a charaeteri/.a<. un of mathematical learning because it Tails to come 
to grips wit.- ti.v jroce^sing of story problems. 

The muU i r i« • .nodi 1 presented c.irliur in ihio paper orovl(i<\<, a boiler 
account of dar i. Within this framework, U:e results of the prese; I 
study can be -*.<pl; r.ied by^two assumptions; the lirst is that both categorization 
and translation are more difficult foiVslory l ban lor formula problems; 
in story problems it is less obvious whaj kind of problem is present and, 



once that is decided and an equation retrieved, it is less obvious which j 
•numbers go where. The second assumption Is tnat .external connectedness, 

linkage arnon^ equations* and e\^lJii\at\^^o\ equations — qualities primarily 
•present in the elaborate hi,;li-oxpJLanatory text — in some combination facilitate 

f i 

categorization and translation. 

From these assumptions, it follows direct Iv that subjects in the hi^h- 
t ».;roup should be less affected by pro!)] em type than those in 
:he^standard and low-explanatory groups »' should be more Likely to <$olvc 
problems when they !ud retrieved tlie appropriate equation and should be 
aided less by the presence of key words in selecting appropriate equations 
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for^story problems. All three of these confluences of the .mode ^tpre supported 

by the data. First, while the standard and Lo\v-e>:ph,aatery proup?^ performed 

significantly loss well on story than on formula problems, the hi .ih-e.\p Lanatory 

* group performed about tfic same op. both. Seconds l"uionu and strict criterion 

\ 

scores w^rc further apart for th • low-explanatory <rad standard groups than 

for the 'high-explanatory group, supporting the assumption <\\at translation 

of the story was more difficult for the first iwo groups. Third, ^he 

nonexpl anatory groups performed a«* well as the hi/h-.^;p 1 an u< rv grjiup only 

on story problems in which the wording unambiguously r<.qut:o*» a particular 

operation. The advantage of the high-oxp lanatory gr nip is nueh larger for 

other storv problems. An examination oi- the Linds of errors siibj^pts made 

on individual problems provided additional evidence about (lie ctfpets of 

text. The performance of the high-explanatory group was less af Ldetcd by 
; * * * i 

the presence or absence of key words, and was loss dep^c.-sed by tfi'c inclusion 

of irrelevant information in the statement of story problems. Si$Ce this 

analysis is post hoc and unsupported by significance tests, the support 

it offers our model is at best tonuous. It does suggest, however, that 

future studies might profitably manipulate the presence* of hey word^and 

irrelevant information, nnd study the interaction with text condition. 

The pouter reliance of i he low-explanatory ami stand ird groups on 

key words /parallels a result 'reported by Chi, boltovich, md G laser (iK>Le 

2) who found that novices learning physics were more 1 dependent upon sQch^ 

surface features than were e:;nerts. They note t hat. both rro'tps may use the , 

same set. of key words but "the actual cues ii-.cil hy tlie vy^p^urc not the' 

wor^g, themselves but wh^thov signify. " U\ out studyv^/i.oh \ , cds.may be 

immediate retrieval CMd«, for the low-ts;p lanatory and standard groups but * 

*may signily secondary cues such as intersections and unions for the high- 

! 
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efcpl ana tgry/ subjects. The present work complements studio comparing expert 
» 

3nd novices. Although al ! our subjects were novices, those Uiu t ;ht yith 
the high-expLanatory text ditfered from other subjects in their processing 

c 

of storv problems in wiys si.mi.Lar to tliose i;i which experts have been r.hown 
to differ from novices. If the goal of Instruction is to move the novice - 
along the path toward expertise, the role of explanation would seem to btf 
critical. ' ' 
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Table 1 

Formulas Presented in .the Three i'exts 

V 

(1) P(A or B or C) = P(A) + P(B^ + P(C) 
(A, i5, and C are nutualiy exclusive) 

(2) P(A or B) = P(A) + P (13) - P(A and B) 

* ** 

(general case) 

(3) P(A and B and C) - P(A) x P(B) x P(C) 
(A, .B, and C are independent) 

(4) P(A, B , B, B, A) = P(A) x P(8) x P(B) x P(B) x P(A) 

(5) P(A given B) = P(A and B>/P(B) 

(6) «P(A and B) - P(-A «ivcn B) :< P(#) . - 



/ 
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[Table 2 



Resp6n*e£ t^oi^hch Fn nation 
Problems, Pooled over Days 



Stpry Data 
Equation 



Group 


1 


• > 


3 


4 


5 


6 


Mean 


Low- Explanatory 


.531 




.547 


.406- 


.203 


. 5 94 . 


.409- 


Standard 


• C / "7 

.547 


.219 


.531 






. oUV 




High-Explanatory - 


- .828 


,297 


.563 „ 


.53! ' 


.359 


.625 


.534 


Mean 


.635 


. 229* 

» 

I 


.547 


.396 


.260 


;609 ; 


.'446- 






I 

Forln;uJa Data 












• 


i ' 

rjnuat ion 
i * 










Group 


1 


2' 

t 


3 


4 


5 


6 1 

c 


Mcan 


Low- Exp 1 a'n a t o r y 


.750 


.484 


.813 


.750 


.594 


.453 


" .641 


StandSrd 


.578 

< 


.578; 


.781 


.766 


.719' 


.453 


. .646 


High-Explanatory 


.750 


.391 


.594' 


.672 


.422 


.484 


.552 


Mean 


.693 


.484 


.735 ', 


.729 


.578 


.463 


.613 



> 
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Statistical inference is part of the research methodQl^g^ of many 
disciplines. It is not % surprising, therefore, to find intrpductory statistics 
courses required, or highly recommended, in the undergraduate curricula of 
such diverse units as departments in the social and biological sciences, 
and in schools of education, business, public health, and agriculture, A 
basic element in such courses is probability: typically, the addition and 
multiplication theorems, 'the binomial theorem, and possibly conditional and 
joint probabilities Given the importance of statistics in general, and" 
probability in particular, an understanding of the consequences of. various 
approaches to .teaching this material is clearly desirable. The need for 
research on the teaching of probability and statistics becomes even clearer 
when we note the considerable evidence that most adults do not understand 
probability and, in fact, have 'pronounced misconceptions " (Tversky and 
Kahneman, 1974), m *v 

In r the last decade, several investigators (Mayer, 1974;/Mayer and 
Greeno, 1972; Mayer, Stiehl, and Greerio, 1975; Myers, Hansen, Robson, and 
McCann, 1982) have varied the relative emphasis placed upon rote and con- 
ceptual learning in teaching elementary probability. The most consistent 
finding in all of these studies is that neither methods which emphasized 
rote learning of formulas nor those which emphasized understanding of 
concepts prpyided a uniform advantage of post-instruction tests. Perfor- 
mance was a function of both the method of instruction and the type of 
problems. 

These results are based on group means for sets of problems. Because 
of this, they provide only limited insights into th$ processes involved 
when novices attempt to solve efementary probability problems. To further 
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our understanding of those processes, we have recorded clinical interviews 
with individuals attempting to 'solve problems after brief exposure to a 

. text which presented, and attempted to explain, a few basic probability 
formulas. Some of the results of those interviews will be presented and 
discussed in the next section. We have also returned <^£o the data originally 
collected by Myers, et al., (1982) to perform a more molecular analysis. 
More precisely, we have examined the types of errors made on individual 

'problems by subjects taught by texts whicli' differed with respect to the 

relative weight placed upon rote learning and understanding of. probability 

formulas. Those error analyses also will be considered in this article. 

♦ 

v PROTOCOL ANALYSES 
Method ' 

Subjects . Nine undergraduate students at the University of Massachusetts 
each received six dollars for their participation. None had previous exposure 
to probability or statistics. 

Text material . The text was 14 pages in length, and contained concepts 

and equations pertaining to elementary probability. It explained such terms 

as mutual exclusivity and independence, and presented six equations, each 

of which Was prefaced by an example. The equations are shown in Table 1. 
» 

Probability was defined as a relative frequency of events, and examples 
emphasized the relation between answers and counts of the number of ways 
that various events could occur. Equations were developed in terms of 
their relation to other equations; for example, the text noted that the 
probability of the union of two mutually exclusive events (Equation 1) was 
a special. case of the probability of the union of any two events, (Equation 
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2). The text used pictorial aids, such as Venn and branch diagrams, to 
explain why an, equation was applied in a . particular example. 



Insert Table 1 ; about here 



Procedure , Subjects were -tested individually. They were instructed 

to read the text at their own rate, trying to 'understand the explanations 

and examples. They were permitted to take notes, but /were informed that 

they would not be allowed to use ttiera during the subsequent problem solving 

period. Most subjects required about 30. minutes to read the text. At the 

conclusion of the study period , ^subjects were given two minutes to study a 

- y 

list of ,15 words unrelated to probability and were then asked to write down 
as many words as they could* recall on a blank sheet of paper. Upon completion 
of the interpolated task, subjects participated in a tape-recorded interview 
in which they were instructed to "think out loud 11 as they attempted to solve 
a series of 16 problems related to the text. A typical interview lasted 

approximately 1 hour, (Three subejcts who required that much time for their 

» \ 
first few problems were not given all 16,) 

When subjects first considered 4 problem, the interviewer was non- 

directive, allowing subjects *to freely answer and explain their reasoning. 

If subjects encoutered extreme difficulty -with a problem, the interviewer 

provided direction by such means as questioning certain aspects of the 

8ubject f s reasoning and encouraging the subject to pictorially represent * 

the inf ortmitioi) in the problem. The interviewer also reminded subjects to 

think aloud when they fell silent, 
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Problems .' The problem set consisted of two story problems for each of 

the six equations in the text, as well as four "conceptual 1 ' problems which 

were included as an alternative means of taping subjects 1 understanding of 

the text. The solution of these last four problems required subjects to 

reason beyond the information provided by a formula. - ■ 

Results and Discussion • V 

Before considering the, problem solving processes, as inferred from 
*■**■ 

protocols, it may be useful to have a sense of the level of ^performance^". 
Answers were scored correct if correctly set up in all respects. Subjects 
were not required to calculate the numerical result. The proportion correct 
for the six subjects who completed the problem set was .9Q_on story Rfroblems 
and .79 on conceptual problems. The three remaining subjects attempted an 
average of 5.33 story problems with a mean proportion correct of .56. 

In examining the interview protocols, we focussed on two questions: 
How did subj'ects attempt to categorize problems? How helpful were th^ 
diagrammatic aids provided in the text?" We/nov/^turn to these matters. 
§ Categorization of problems . Categorizat/ojt. is a crucial stage in 

problem solving (Hinsley, Hays & Simon, WlT). Cognit^fe psychologists 

./ ' 

have suggested a number of ways in which ^Instances of a category may be 
classified, among them categorization on*/the basis of critical features 
(Bourne, Dorainowski, and Loftus, 1979) and categorization by, analogy to 
previously experienced instances o'f the category (Brooks, 1978). Our 
subjects exhibited both of these processes. We will consider each in turn. 

- Subjects frequently used key words such as "and, 11 "or," or "given" as 
cues to categoriz&tion of a problem. Following are two typical protocols. 
The first demonstrates how a subject decides between addition and multiplication: 

ERIC o. 

idflliiliTlifrTlflid <./ i i 



5 



S:...If I was relating different events by an "and 11 instead 
of an "or," then I would multiply the ^probabilities, whereas 
if they were "or" - then I would add the probabilities. That's 
the way I looked at it while I was doing it. So I ! m trying to 
aee whether these are "ands" or "or§." . 
Other protocols illustrate dependence on the key word "given." Note that 
the subject in the following excerpt is somewhat confused about exactly 
what goes into the numerator of the ratio of probabilities: 

S:...I remember at the end-there was a formula tha^ showed you 
how to calculate someting given something else. And it' had to 
do with a ratio — a ratio of these two — added together — ...A / 
given B — I remember that part — was equal to the ratio where 
^ the numerator was A plus B. 

» ♦ 
Reliance on key words is only partially, successful. Even if the 

problem is categorized correctly, the » key w v ord strategy provides no, help in 

1 V 
substituting the correct values from the problem into the equation. The 

key word Strategy often does not unambiguously distinguish between equations 

i 

(for example,' between applications of Equations 1 and 2); and it is 
inapplicable when the key word is not explicit in the text. We shdll see ^ 

evidence of- -theSeTUifflculties- in -tile section on "error- analyses.. . v 

Finally, even when applicable, the key word strategy is subject to 
misuse. For example, protocols demonstrated frecjuerit confusion about which 

% 

word signified multiplication and which addition. We shall see evidence of 
these difficulties in the section on "error analyses. 

Although % key words were clearly important to subjects, tiiey also 
casionally verbalized their recognition of, the similarity of a problem to 
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arv example seen in the text. *Following is an excerpt from a protocol of a 
subj'ect , who, in trying to solve a problem, notes that he had seen an 
analogous example in the text: ? 

Subject(S); Okay, 50% is under 40, 60% is male, 20% is both... j 

. . This one seems to be like that ace of hearts one (an 

example from the text)''. 
~" Instructor (I) : Why do you say that? 

S: . Becau^, it overlaps-there 1 s part of it that applies 

* . to both 40 and male. So under 40 or male — I know I 

subt r act some thing — ■ 
I: What do you think you should subtract? 

S: I subtract this. ^ 

. I: , The 20% under 40 and male? 

S: • Uh-huh ' 

I: Why 4 do you think you should subtract that? 

S: „ Because I remember the one — you were supposed to 

subtract the ace of hearts— ;that was the overlap. 

So I think tftese involve addition. 
I: Why do you think that? 

S: Just the way I remember it in my mind — the problem 

was laid out in the text. ..It was the chance of 

r 

drawing an ace or a heart and they got three chances 

of drawing an aceC~13 of drawing hearts and i^ras 

minus one for the aWof hearts. So — it doesn't 

« 

come out to the same answer that they gave. 
I: Dp you know why? 
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Sr I^think it came out to(l7. Oh, maybe I should put 

ir aces* 

I; Does that make sense to you — 4*aces and 13 hearts?^ 

S: * Yeah, ! cause» that's the whole thing. And that'^s 

just minus 'the overlap. So now I'll go up here and 

i 

do 'this up here. Four aces, thirteen hearts, 50% 
under 40, 60% male, 20% under 40 and male; so that 
r would be .5 plus 60%, male minus .2 equal to ,9. 

That's a lot. 1 

\ 

Protocols such as this demonstrate that subjects were able to go beyond 
superficial features and extract classification cues. The subject in the 
example noticed the intersection in the problem, used this cue to recall an 
example, and analyzed the example 'to arrive" at t^e solution. In short, the y 
subiect made use of conceptual relations between the problem and the example, 
^ rather th^n relying on memorized associations between surface features of 

the problem statement (e,g,, the word "or") and a formula. 

r 

Y/ The tole of diagrams in the text . Venn and branch diagrams were included 
with the presentation of each equation in the text in order to aid subjects 
in understanding the rationales for these equations, and to provide a frame- 
work within which to solve problems. Comments by subjects in th£s study 
and in a previous one (in which subjects were asked to think aloud as they 
worked through a question^-and-answer version of the text) ^indicate that the 
diagrams did help 1 subjects understand the text. However, they were less 
successfully used in solvin^t^st problems. Although most subjects attempted 
to use at least' one of these. representations at some point during the problem 
solving session; they did not attempt^^use them very frequently, . 
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Mistakes' involving Venn diagrams were typically £ue to confusion ^bout _ 
how to construct and label the diagram, rather than in inability to interpret 
a correct, representation . For example, one subject raad^ tfcree attempts to 
represent the information given in the problem presented in footnote 1 y-^H 
the first twcy attempts, the subja£t was unable to represent the information \ 
that 60% of the' city's population was ntale. This was because he focussed 
on the probability of being under forty and its complemfetrt, first dividing 
a single circle into areas of "over" and "under 40, n and then drawing a 
diagram containing an intersection of "over ahd under 40." In the third 
attempt at a representation, ^thejmbject drew two' non-overlapping circles 
for "over" and "under 40," with a third circle, labelled "males," over- 
lapping each other. This diagram,* however, contained an area which 
supposedly represented males that are not ovej^or under 40. 

Misrepresentation involving branch digrams seemed to occur most often 
because subjects lacked the ability to move from a representation involving 
relatively few concre^, countable events (as given in the ^text) , to one 
that involved more abstract^probabilities of events that ^re more difficult 
to count,. While the subject in the following example did not actually draw 
a branch diagram, the protocol is relevant in that it shows the subject 
attempting to obtain the same .result -that a branch diagram would yield; 
namely, the number of events in the • sample space and the number of events 
of interest. The following is an exceprt of a subject attempting to sol*ve ' 
the following problem: 

"On a multiple choicp- test with 4 choices for each item, a 
student guesses. What is the 'probability that he is right 
on items 1 and 2 but wrong on 3 and 4?" 
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Subject(S): Probability of bein^ "right is one fourth, .25. 

And wrong would be v 75. 
Interviewer (I) : Okay, how did you get those two? 

S: v f Well/'out of the four choices for each item, you. get 

*to choose one — so that would be one out-, of four 

chances — you have one chaqf^ of being right — which 

i 

would be one to four which would be .25. Arid to be 
wrong — there are three wrong answers and. one righr 
* , answer, so four questions with four choices 

^ " each — there's sixteen possible choices* - ^ 

* • i 

90 sixteen possible choices, four could/fca- * Igh I ' , ' 
• ** 

twelve could be wrong. Chances of/getting them all 

right would be .25. 

I: The chances of getting the first one right, the 

second one right, and the third and fourth is .25? 

S: Yeah. 

» • 

I: Okay, why do you say that? r ^^_ 

S: Because, out of the -sixteen possible choices for all 

four questions, there are only four rigjjt answers, 
so it would be four sixteenths — oneXfourth — I'm 



going to xake a wild guess at this^fie — and say, it's 
one eighth. . 
I: - Why do you say that? 

S: Because t>o get them all right is — it's .25. That's 

„ x the chance you get, one quarter chance. So you're 

halving— wait a second — you's halving that. 
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Why are you halving that? <? 
S: ^ ^ Well, you're only getting two right — but you're only 

getting two wrong, too, — The chances are better for 

c 

getting things wrong — I don't know. * 

This subject has made a fairly reasonable attempt to obtain the answer as a 

relative frequency. The major error occurred in defining an event as one 

of sixteen, alternative atiswers , (four for each question), instead as one of 

256 pos*Slbie sequences of answers. (The text had.not explained that 

probabilities *may be assigned to segments* of a branch diagram in order to 

simplify it. When all segments are, equally weighted, the correct diagram 

has 25fe~ possible paths, nine of which satisfy the conditions specified in 

the problem.) This subject seems to have been aware that her representation 

was incorrect, that it failed to compensate ^br the greater probability of 

ing. incorrectly. However, she was unable to us# this knowledge correctly 

* *" 

subjecT~~"5eBms to have been prepared to think in terms of a fairly stria 11*, 
countable sample space, but unable to extend this thiniirig to situations in 
wh\Lch very large sample spaces or abstract probabilities are involved* 
Although the use of branch diagrams by our subjects was % infrequent, this 
difficulty wak typical oh those occasions when they were employed. 

~* 

ANALYSES OF" ERRORS 



In the study just described, we employed ortly one text. An earlier 

j 

study by Myers et al., (1982)' provides a basis for comparing the effects of 
instruction by that text (which we will, call "explanatory") with the effects 



of* instruction by texts which provide little explanation, or integration, 
of the equations taught. In this section, we provide a more molecular- 
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analysis of the data than'is available in the. earlier article, ,JBy examining 

c 

the kinds of errors n&de on individual problems under different:' text conditions 
wfe arrive at some sense of the way in which different Instructional modes 
influence what is^leamed and how it is used in solving probability problems, 

, ■ y ' * 

Method v . " 

Subjects . Forty-eight 'A^lunteers from an Introductory Psychology course 
w£re randomly assigned to one of three text conditions. None had previous 
exposure to instruction in probability or statistics, " „ 

Texts » There wre three text conditions: The explanatory text was the 

same as the text used in the protocol- study described in the preceding section, 

A second text was labeled "standard 11 because it seemed similar to mdily 

introductory Statistics textbooks currently in use; it treated probability 

as a measure with certain well defined properties (e,g, , limits of one and 

zero) and used familiar real-worlM examples (e,g,, the probability of rain 
f 

tomorrow) but did not explain probability in terms of counting processes. 
Nor did* the standard text ptovide any explanation or develop relations 
between equations, A loW-explanatory text was even more abstract than th 
standard text; aside fi?bm an initial reference to dic'e throtfijig, examples 
were stated in terms of such quantities as P(A) and P(B)-, All three texts 
preserited an. example followed by the appropriate equation, as well as 
definitions of terms ^uch *as independence and mutual exclusivity, A more 

defiled presentation of the material'Mn, the three texts may be found in * 

1 ' > 

Myeirs efal. t (1982), . . 

i Problems , The 24' story problems included the 12 useci in the protocol 

*i ' r + 

analysis study and 12 others that were-similar , There were four problems 

' s ' V 

testing knowledge of each of the six equations presented in the texts* 

1 * • $ i 



Procedure . Subjects were te'sted individually. "Theywere instructed 
to read the text booklets silently and at their own rate, trying to under- 
stand the 'explanations and examples. The standard and low-explanatory 
groups, whose texts-were four pages in length, "were told that thej^ would 
have \A "minutes to finish; the explanatory group, whose text was 14 pages 



xrl^ length, were given 25, minutes. These limits, based on pilot subjects, 
were -imposed to discourage aay individual from either quickl^* scanning the 
texts, or taking an inordinately long time. All subjects finished 'reading 

4 «* 

y * * " 

their texts in the* given tijne period for their group. Subjects in the low- 
explanatory and standard groups tended to rate the study time more favorably 
than subjects in thd explantory text' condition, but this difference was not 
significant 

At the completion^ of the study period, subjects ftere. given the same 

interpolated task used in the protocol study. Upon completion -of this task, 

they were given a test .consisting^ of half of the problems. There was one 

problem on a page with space to^ do any work which the subject felt might 

help in the, solution. Subjects were given as much time as they required to 

complete th^test; most required about 30 minutes., cUpon completing .the 

problems, subjects were asked, to return 48 hours later. On the second test 

day, subjects were given the second form 6f the text, consisting^ of 'the 

other 12 problems. The order of the presentation of the test forms was 

counterbalanced . t H 

• Fdllowing the second performance test, subjects were given as much time 

\ i 
as they required to respond* to kptitude tests^ and to a questionnaire. These 



measures are described in Myers et al., (1982) /'where the result of their 
analyses is also reported. , , 



Results 



\ 

We have chosen tg . focus on, those problems 'testing knowledge of Equations 



1, 3, and 4; the problems within^ these sets revealed the widest variation 
in proportions of correct responses. In considering answers to these problems, 
we ask: why is there taarked variation in proportion correct to problems 
requiring the sam^ equation* for solution? And why is this pattern of variation 
affected by text condition? 

The low-explanatory and standard groupsf performed in similar fashion 
on all problems.* Therefore, inUhe interests of simplifying the presentation, 
we have treated them as a single condition which we have labelled "nonexplanato 

We begin our consideration of the results by briefly noting Table 2^which 



Insert Table 2 about here 




presents proportions of correct responses for each condition for each of 
the 'four problems for each of the rules presented in Table 1. It is/clear 
from Table 2 that .there was cohsiderable variability within sets* of problems, , 
and that the pattern of proportion correct of ten!" differs markedly for the 
two text Conditions. In what 'follows, we will attempt to demonstrate that 
the problem-specific effects are due to elements of the problem presentation 
which'affect the ease of classification and translation, and, that subjects 
in the- nonexplapatory condition were,,raore sensitive to these elements. 

' Equation -1 » 'These four problems provide an example of our thesis that 
subj^e^s in the nonexplanatory condition are sensitive to aspects of the^ 
problem stabai^nt whichl have little effect upon the performarice of subjects 
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ip the explanatory condition. As indicated by Table 2, proportions of 

« 

corxect responses in the nonexplanatory group varied greatly across the 

s 

four problems testing .knowledge of Equation 1, while^ performances of those 
±n the explanatory group are not only much better but also quite stable. 

One reason for the variation in performance in the nonexplanatory 
condition is that — for these subjects — some problems are much more difficult 
to clasrsify than others. There are at least two aspects of a problem's 
statement that can cause such difficulties in misclassif ication: the 
presence of irrelevant information and the absence of key words asspciated 
with use of a particular equation. Both are illustrated by errors made in 
response *to Problem IB. This problem states that: 

"In the National Women's Pingr-Pong League, Los Angeles has a 
.3 chance of winning, San Francisco has a .2 chance, San Diego 
has a .1 chance, New York has a f .1 chance, and Philadelphia has ' fc 
a ,3 chance. What is the probability that a Wait Coast team 
,willwin?" 

Note that this problem contains irrelevant information; the staged 
probabilities for New .Xprk and Philadelphia are unnecessary. Subjects in 
the nonexplanatory condition attempted to use these probabilities; five 
subjects subtracted , l*f,3 from the. correct sum (or from thq product of the 
West Coast probabilities), four others multiplied all five probabilities, 
and two others divided by .1 + .3. Only one subject in the explanatory 
condition attempted to incorporate the irrelevant values into his answer. 
Apparently, subjects who do not understand the rationale for using the 
equations attempt to use any information presented to them. In this example, 
the subjects cited misclassif ied the problem; using. Equations 2, 4, or 5 in 
an attempt to incorporate all* the values stated. 
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Eight of the 32 subjects in thfe nonexplanatory "condition used Equation 
3 rather than Equation 1, multiplying probabilities instead of adding thera. 
It is revealing that the only ot^er of the four problems in which this type 
of error was frequent (nine of 12 subjects who made errors) was problem ID, 
which states: 

"In a large qity hospital in 1978, the proportion of births 
involving twins was .06; for triplets, the proportion was .02; 
for more than triplets, 'it was .01. If wq- define a multiple 
birth as one involving more than a single child, what was the 
probability of a multiple birth in that hospital? 11 
(IB) and (ID) are the only two problems in this set in which the problem 
does not specifically request the probability of something or something. 
It is also important to note that this raisclassif ication error occurred 
only once for Problem (IB) and once for Problem (ID) in the explanatory 
condition. Apparently, the absence of the key word causes difficulty for 
subjects in the nonexplanatory condition. 

Even when subjects in the nonexplanatory condition used the correct 
equation, they may have had difficulty .translating the story; that is, 
correcjtly inserting values from the story into the equation." This was a 
major source of difficulty for the nonexplanatory group in Problem 1A. The 
problem states: 

"A marble is drawn from a jar containing 10 red, 30 white, 
20 blue, and 15 orange marbles. What is the probability of - 
drawing a re^ or white marble?" 
Five subject's raisclassif ied the problem, attempting to incorporate frequencies 
of orange and blue marbles into their answer. Most of the errors, however, 
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reveal a lack of understanding of the relation between frequency and probability. 
Subjects often faii^ to divide the sum £of the numbers- of red and white marbles 
by the total number of marbles, or divided by 100. Note that this problem 
was the most difficult for the nonexplanatory group but the easiest^for the 
explanatory group who had. the relation between frequency and probability 
explained in tFTeir tex7^^ / lFh4^provide s^evide nce of the effects of key words, 
irrelevant information, and translation requirements on the performance of 
subjects in the- nonexplanatory group. The problem states: 

"The probability is .2 that Jimmy will win a race and^ .3 that 
he will finish' second. Whatis the probability that he will 
win or finish second? 11 
Note that this problem contains no irrelevant values, includes the key word 
"or", and involves a simple translation of the stated values. This problem 

) 

evoked the best performance in *he (nonexpJanatory group. 

Equation 3 . The explanatory group did not perform as well on these 
problems as on those testing knowledge of Equation 1. We suspect that the 
rationale for multiplication o£ probabilities is more difficult to understand 
than that for adding probabilities*. 

In one respect, performances on this problem set were similar to those 
on the first problem set: proportion correct was fairly stable over problems 
for the explanatory group and quite favorable for the nonexplanatc\ry group. 
As with the first problem set, it appears that subjects in the nonexplanatory 
cond^tt&n are more sensitive to the statement of the problem than. are subjects 
in the explanatory condition. 

Much of the variability exhibited l)y the nonexplanatory group can again 
be traced to variations in the problems with respect to irrelevant information 
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and translation requirements. Problem 3D presents values that are redundant; 

"In a certain class, there are 80% men and 20% women. Also** 
' 60% of the class are freshmen and 40% are sophomores. If sex 

and ye*c are independent, what is tj"he probability that an 

individual? is female and a freshman?" 
Four subjects in the nonexplanatory group used the wrong equation in an 
attempt to incorporate all four values in their answer (e.g., ".-2 x .4 x .6 
x .8") and one made what w6 would characterize as a translation error, 
multiplying .2 by ,4. In contrast, no subjects in /the explanatory group 
used numbers other than .2 and . £ in their answers. 

The aasyers to Problem 3A indicate a serious translation difficulty 
for subject^ fin both conditions. The problem states' that: 

"The probability v that a man will be alive in 20 years is *4. ( m 

The probability his wife will be alive is .6. Assuming that 

these are independent events, what is the probability that 

he will be altve and she will be dead in 20 years?" 

Of 23 subjects in the nonexplanatory condition who made an error in answering 

s 

this question, 11 made the response ".4 x .6;*" one subject in the explanatory 
condition gave this answer. Nine other subjects in the nonexplanatory group 
substituted .4 and .6 into the wrong equation; five subjects in the explanatory 
group did this. 'It is not clear why subjects had so much difficulty classifying 
this problem. It is clear, however, that 20 of 32 subjects in the non- \^ 
explanatory group, and six of sixteen in the explanatory group, failed to /^ 
translate the stated probability of the woman being alive into the probability 
that she would be dead. We will shortly consider other evidence that 
translation of a stated probability into its complement is more difficult, 
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and more often overlooked in the nonexplanatory condition.. 

In contrast to performances on Problem 3A, the nonexplanatory group 
very yell in answering Problem 3B. This problem states: 
".7 of the members of the state legislature favor a particular 
bill. .6 of the members are Democrats* fLf party and vote are 
independent, what is the probability that the next legislator 
you meet is a Democrat and votes for the bill?" 
Why was this problem so easy for the subjects in the nonexplanatory group? 
We suggest that they have memorized the rule: "Whenever the probabilities 
of something and something jelse are required , ^nultiply their individual 
probabilities. 1 ' In Problem 3B, the^word "and" is present and thei\£ are 

r 

only two possible values to multiply. Fortunately , they are the right two 
values . 

Equation 4 . This equation is the general case of Equation 3 and, 
theref ore,'^Lt^would seem that subjects should score about the same on this 
problem set as on the earlier one. That was the case for subjects in the 
explanatory group; in the nonexplanatory group, however, the average 
proportion correct was .22 lower for the problems testing knowledge of 
Equation 4 than for those testing knowledge of Equation 3. For the 
explanatory condition, the proportions correct were, as in the other problem 
. sets, relatively stable. Also, as before, the nonexplanatory group exhibits 
greater variability of performance over the four problems. 

That the nonexplanatory group had greater difficulty with these four 
problems than with the third problem set may reflect two factors. First, 
these problems typically omit explicit use of the key word "and"; this may 
* cause classification problems. Second?, the probability of the complementary 
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event* is often not explicitly stated; thus, there may be translation • 
difficulties. For example, consider the wording of Problem 4A. 

-# 

fl A baseball player gets a hit in 30% of his times at bat. 

>^What is the probability that he> goes hitless in all four 

times at bat on a particular day, assuming that .the result 

of one at bat has no effect on the result of any otl^ef at bat? 11 

Problems 4C and 4D are similar in that they lack the key word "and" and the 

student must supply one probability by subtracting the 'value given, 4» 1.0. 
*» , — - ' x 

In both conditions, there are errors of -misclassif ication: for example, 

t 

averaging over problems 4A, 4C, and 4D, five of 32 subjects in the non- 
^explanatory condition added instead of multiplying values, and 2.33 of 16 
did this in the explanatory group. There were also attempts to subtract 
and divide probabilities. More striking, at least in the nonexplanatory 
condition, were errors of translation. Averaging over the three problems, 
11.67 subjects in the nonexplanatory group made errors .of translation; for 

H 

example, giving .34 as an answer to'Problem 4A. Only 2.33 subjects made 
such errors in the explanatory group. 

* / 

, The nonexplanatory group performed reasonably well on Problem 4B which 

states: - ✓ * 

^ M A*hbckey team wins with probability of .5, loses with prob-„ 
ability of .3 and and ties with probability of .2. What is 
the probability .that in the next 4 games, the team wins, then 
ties, then wins, then loses? 11 
The st&tement of this problem seems to more closely parallel the statement 
of Equation 4 than does that of the other problems in .this set, ~thus reducing 

r 

the likelihood of misclassif ication. Furthermore, all the required prob- 
abilities are stated; translation difficulties are presumably minimal. 

ERTC '43 • 
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GENER«UT)ISCUSSION. « * 

In order to examine more closely* the effects of an explanafory text on 
subsequent problem solving perf ormancepwe have recorded interviews with 
students as they worked on individual problems. In addition, we have analyzed 
the types of errors made by subjects under dif fer^ft^^ext conditions to 
determine whether these errors would form anj( recognizable pattern. These 
interviews and error analyses provided ^eme^ansights into both the successful 
fid unsuccessful strategies our subjects usect^and uncovered several common 



sotrrces of difficulty underlying their attempts to solve elementary prob- 
ability , problems. , * f 

The protocol analyses revealed that our subjects relied primarily on 
critical features, or n key words," in the problem statement in order to 
categorize the problem. They were also able to categorize some problems on 
the basis of an analogy to the example provided in the text. Both the 
interviews and rhe error analyses revealed that the key word*stragegy is 
only partially successful in categorizing probability problems because it 
does not always unambiguously differentiate between equations and leaves 
the subject at a loss when the key word is not explicit in the problem 
statement. In addition, even if the problem is correctly categorized, the 
key words are of little or no help in translating the values in the problem 
into the, variables of the appropriate equation. Finally, subjects 1 protocols 
indicated frequent confusion about which key word signified which operation. 
This may be the result of the words u a(id," "or," and "given" having different 
conncftations in the context of probability problems than in their more common, 
everyday use'witft which the subjects are more familiar. >For example, subjects 
often wrote ff P(A + B) when the problem required the probability of some- 
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thing and something else. Although it is not difficult to understand these 

errors (it isya^t considered improper to say "two and two equal four"), 

this connotation of "and" makes it a less discrirainable ciie for multiplication. 

On the other hand, subjects were more likely to arrive at the correct 
solution when they categorized problems by analogy to the example presented 
in the text* This sort of analogical reasoning seems less prone to the 
pitfalls (QfT associated with the 'use of key words and thus desirable to 
encourage. An important faCtot. in developing this 'approach to categorization 
may be an emphasis on explanation within texts. Although the subjects 
taught by an explanatory text in the protocol "study did employ key words in 
solving problems, they are much less dependent upon those words than subjects 
taught by texts that place greater emphasis on rote learning of formulas. 
This was amply demoKstrated in our analyses of errors made by subjects from 
N£v/o text conditions. Subjects in the nonexplanatory text condition made 
more classification errors on problems in which the key word was missing 
from the probj^ statement than on problems in which the key word was 
explicitly stated. For subjects in the explanatory text condition, the 
proportion correct was about the same regardless of the presence of key 
words; that is, their performance was relatively stable across problems *\ 
testing knowledge of the same equation. 

One limitation of our explanatory text may be that the text .e^o^nples 



we 



re different from those in,' the text. Several subjects indicated that 



they understood the text but found the problems ia-ibe^-e)t^nore "abstract" 
than the examples in the text. This may be because the examples in the text 
used countable sets (e,g,, the number of marbles of a certain color, or the 
number of caifds of a certain value) while those in the test .most often J 
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presented proportions and percentages. This observation points up .the need 
for more research on the role of examples in the, text. Based ^>n subjects' 
comments about the need to have test problems more like text examples, we 
suspect that it wiLl be important to identify the dimensions along which 
problems can vary. Our restilts suggest some of these dimensions: the 
presence or absence of countable events, of key words,, of irrelevant infor- 

U 

mation, and of quantities that must be transformed in order to solve the 
problem. ^ If examples are to serve to illuminate concepts, and to provide . 
an adequate ; base for analogical problem solving, those used in instruction 
should vary along the important dimensions. Such variability, rather than 
the sheer number of examples in the text, may*we^rl be the critical factor. 

The protocol analyses also revealed that our subjects seldom used Venn 
and branch diagrams in solving text problems, although subjects' comments 
in4icated that these diagrams did help them understand the accompanying 
examples in the text. Attempting to represent the problem statement in a 
diagrammatic form proved to be a source of considerable difficulty for the 
subjects we interviewed. These findings indicate that understanding of the 
representation of an example in the text does not necessarily translate 
into the ability to represent a new problem. To the extent that such an 
ability is important, practice at representing problems is a critical part 
of instruction in probability that is generally overlooked. How to implement 
such practice is an important question to be Researched. It is not clear 
that the representations we have used, which are those most typically found 
in probability texts, will function best. Recent texts have incorporated 
more'concrete diagrammatic aids, k for example, pictures of individual cards 
and alternative outcomes of throws of dice (Freedman, Pisand, and Purves, 
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1976; Pagano, 19{U<). Systematic research on the effectiveness of various 
modes of representing probability problems seems in order. 

m 4 * 

T^e error. analyses revealed that variations in test performance on 

> 

problems requiring the same equation, for solution' were due to various 
elements of the "problem presentation which affect the ease of classification 
and translation, Subjects in the nonexplanatory text condition were more 
sensitive to those elements than subjects in the explanatory text condition* 

For subjects in ttv nonexplanatory text .condition, the presence of 
.irrelevant information (values not required for solution) caused errors. 
These subjects tended to incorporate the extra values in their answers more 
frequently than subjects in the explanatory text condition, which indicates 
that they may have a very limited understanding of the rationale underlying 
the use of each formula. If this is the case, it appears that .the nurabllr 
of variables in the problem, rather than. what the problem actually /^sks for, 
will .tffcctata which formula is retrieved and \sed for solution, ^n^ddTtionl 



/ the types of errors made on certain problems ce^t- ai T ii ng additi o nal infor ma-fe-i 
revealed that many subjects in the nonexplanatory text condition lacked the 
fundaraeTrtal concept of probability as a relative frequency, even though 
^ey c ould ^coirre ctly apply formulas to simpler problems, 

The error analyses revealed two types of translation errors both of 
which occurred more frequently in the nonexplanatory condition. The first 
type was revealed by problems which required the subtraction of one of the 
given values from one. The most frequent error on these problems involved 
inserting the stated value, rather than its complement, into the equation. 
The second type involved the translation of a verbal statement in the problem 
into a probability required for the equation. These errors occufred on 
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problems which .Involved countable events. For example, on one problem testing 
Equation 2,^ several subjects translated "the probability that* a randomly 

Selected person has a birthday on the first of any month" into the' value 
1/L2 in their answer. It is clear that there is some counting process under- 
lying this translation, anfl-that subjects who make this kind of error do/*^^. 
have some sense of^probability as a relative frequency. Nevertheless, they 
fail to take into account the entire sample space in their answ.er. ; . „ 

Research on translation skills and Various sources of translation errors - 
(e.g. t Clement, 'Lockhead, & Monk, 1979; Paige and Simon, 1966) has indicated 
that translation of story problems into mathematical equations is a very 
common- source of difficulty £or many students. Although translation skills i 
ar£ essential to understanding and successful problem solving, they are 
generally taken £OT-gr«nt*ed in instruction. Our analyses suggest that 
practice at translating probability problems should be an important part of 

' instruction in probability, ^tjis^clear that a major factor in developing 
translation skills is an emphasis on explanation of the basic concepts 
required for successful translation -over a wide range of problems. 

The explanatory text used inr bo<th this and the prtftTiOus study defined 
probability as a relative frequency of events, and presented examples which 
emphasized the relation between answers and counts of the numbers of ways 
that various events could bccur. Myers etf al. (L982) demonstrated that 
this emphasis on explanation did ^facilitafce the translation * of story problems. 
They used both a strict and lenient scoring criterion in analyzing story 
problem data. The strict criterion required that the solution be correct, 
in all respects, while the leni^At criterion accepted as correct the 'appro- 
priate equation, regardless^ whether the values were correctly inserted. 

7 ' * 
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The difference between the two scores thus provided an index of translation 

* * 

*J 

errors. The explanatory group had fewer translation errors; strict and 
lenient criterion scores differed much ]es's in the explanatory text condition 
than in the nonexplanatory text condition (See Myers, et al. f 1982, for a 
more detailed discussion of the contrast between text conditions with respect 
to the strict-lenient difference). 

The findings of the protocol study and the error analyses reported 
here complement the results of studies using similar methods to investigate 
the processes underlying students 1 problem solving performance. Chi 
Feltovich, &'Glasser (1981) investigated the categorization of physics 
problems by experts and novices. Like our subjects in the nonexplanatory % 
text condition, the novices they interviewed relied primarily on the surface 
features or key words in the problems, whereas the experts tended to categorize 
k the problems according to the underlying physics principles. ^ .Although all 
of our subjects were novices, those in'the explanatory. text condition differed 
from the others in their processing of story problems in ways that; are similar 
to those in -which experts differ from novices. s ' 

Clement (1979) found that engineering students were particularly likely 
to apply the wrong formula to elementaty physics problems containing extra 
information. Moreover, his analyses of^student problem solving protocols 
indicated that a student's understanding of elementary principles of physics 
can be very weak, even though the student can remember and manipulate ^relevant 
formulas. Similarly, our subjects in the nonexplanatory text condition made 
more translation and classif icaticj^rrors t^an the others, even though there 

was some indication that subjects who read the nonexplanatory 1 text remembered 

» 2 

the formulas better. * /~ 
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The clinical interview method, and the analyses of the types of errors 
made on paper and pencil tests, have shown that the stragegies used and the 
errors made are seldom unique to a. single individual; rather ,' those strategie 
and errors form specifiable patterns apross subjects wti*ich reflect the degree 
of their understanding of the concepts under study. These methods are 
increasingly being used as research tools in investigations of human problem 
solving performance. In the present study, they have bro.ught us closer to 
a precise definition of what it means-.to understand elementary probability 
and to a specification of the processes such understanding might affect. 
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Footnotes 



The research reported in the paper was supported by a grant from the 
National -Institute of -Education, NIE G80-0126, and by a grant from the 

I 

National Science Foundation, BNS 80-01181. Reprint requests should be N 
addressed to: Dr. Jerome L, Myers, Department of Psychology, University of 
Massachusetts, Amherst, Massachusetts, 01003, 

> 

*Thef problem states: "50% of a city's population is under*40 years of 
age. 60% of the city's population^ is male. 20%_is under 40 and male. 
What is the probability that the next person I meet will be under 40 or male 



2 

Myers, et al. (1982) found that subjects -in the ^nonexplanatory text 
condition performed better than subjects in the explanatory condition on 

■mm- 

formula problems such as ft P(A) = .4 and P(B) - .3. If A and B are 
incompatible events, what is P(A or B)? ft , but this difference was not 
significant. ''However, they^also reported a follow-up study in which _ 
subjects in the nonexplanatory 'condition showed significantly better recall 
for formulas. * . L • 
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Table 1 

Formulas Presented in the Three Texts 

(1) P(A or B or C) » P(A) + P(B) + P(C) 
(A, B, and*C are mutually exclusive) 

(2) P(A or B) =» P(A) + P(B) - P(A and B) 
(general, case) 

(3) P(A and B and C) = P(A) x'pTBI x P(C) 
. (A y JB^and C are independent) 

' (4) P(/^ B, B, B, A) = P(A) x P(B) x P(B) x P(B) x P(A) 

(5) P(A given B) = P(A and B)/P(B) 

(6) P(A and B) = P(A and B) x P(B) 
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Table 2 y 
Proportion Correct for Each Group for Each Problem 



NONEXPLANATORY 



PROBLEM 

A 
B 
C 
D 



A 
B 
C 
D 



A 
B 
C 
D 



A 
B 
C 
D 



A 
B f 
C 
D 



A 
B 
C 
D 



TOTALS 



TOTALS 



TOTALS 



TOTALS 



TOTALS 



(n = 32) 

P(C) 

.4063 
.4375 
.7188 
.6563 



TOTALS 



.5547 

.2188 
. 15,63 
.2500 
.1563 
.1953 

.2813 
.8438 
.5625 
.5000 
.5469 

.1563 
.5625 
.1875 
.3125 
.3047 

.1875 
.3125 
.0625 
.1875 
.1875 

.5938 
.5938 
.5625 
.6563 
.6016 



EXPLANATORY 
(-n = 16) 

P(C) 

.9375 
.7500 
.8125 
.8125 



.8281 

.1250 
.4375 
.3125 
.2500 
.2813 

.5000 
.6250 
.5625 
.6250 
.5781 

.5000' 

.5000 

.4375 

.4375 

.4688 

.4375 
.3750 
.1875 
.4375 
.3594 

.7500 
.4375 
.6875 
.6250 
.6250" 



TOTALS 
(h = 48) 

» 

P(C) 

.5833 
.5417 
.7500 
.7083 



.6458 

.1875 
.2500 
.2708 
.1875 
.2240 

.3542 
.7 708 
.8438 
.5417 
.6276 

.2708 
.5417 
.2708 
.3542 
.3594 

.2708 
.3333 
.1042 
.2708 
.2448 

.6458 
.5417 
.6042 
.6458 
.6094 
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ABSTRACT > 

4 

Subjects with no prior knowledge of probability read one of three texts; 
the texts varied in the degree of explanation and integration of basic con- 
cepts of elementary probability. Subjects in the high-explanatory text con- 
dition did equally well on both story 'and formula problems, falling below 
the formula performance of the other groups and above their story performance, 
A detailed analysis of error protocols for each story problem showed marked 
differences in which problems caused the most errors for the different groups 
and in the kinds of errors made. The high-explanatory group appeared to be 
less dependent upon key words, better able to filter irrelevant information, 
and more likely to insert values correctly into formulas . Thus, even with 
novices, qualitatively different knowledge structures can be produced by 
texts which vary in their emphasis on understanding. 
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Th<* Role of Explanation in Teaching 
1 6 Elementary Probability / 

Jerome L. .Myers 
/ Department of Psychology 

University of Massachusetts 
Amherst, MA, USA 01003 

Parents, educators, and mathematicians have been involved in recurrent 
debates about the relative merits of understanding mathematical concepts 
and rote mejiorizii^g computational formulas. In principle, such conflicts 
should' be resolvable through research involving manipulation of the degree 
to which students understand "mathematical material-and observation of their 
performance on 'tests of mathematical knowledge. In actuality, we encounter 
difficulties because we lack-both a precise definition of mathematical under- 
standing and a specification of the' processes such understanding might affect. 

We have begun a research program to investigate the role of understanding 
in learning elementary probability <by university undergraduates. Although 
we still lack a precise definition of understanding, we have identified three 
dimensions which have provided the basis for constructing short texts. The first 
such dimension, external connectedness, corresponds to the degree to which a 
text explicitly relates the concepts and formulas to.be learned to the real- 
world knowledge the student presumably brings to the situation. The second ^ 
dimension might be , termed "internal connectedness," the degree to which rela- 
tions among formulas and concepts are explicitly developed. For example, 
an equation to be learned may, 'or may not, be presented as a special case 
of a formula presented earlier in the text. The third dimension is explana- 
tion, the degree to which a rationale is provided for a particular formula* 
This refers not to derivations, but to intuitive justifications for procedures 
taught. For example one of our texts uses branch diagrams to explain why 
tfie probability of the joint occurrence of two independent events is a product 
of the marginal probabilities. - — 



We will describe the texts constructed within this framework more fully 

in a later section. First, however, we wish to consider certain processes 

which should be involved in solving Rjrobability problems, and the ways in 

which these may be affected by the degree of understanding imparted by the 

text. We assume three stages.: categorization of the problem, retrieval 

y 

of the formula apprppriate to that category, and translation of the profrfjSn — 
substitution of values from the statement of the problem into the retrieved 
formula. In general, understanding should aid both the categorization and 
translation of problems. Understanding may aid or hinder retrieval of formulas 
from memory. On the one hand, the linkage of ideas and formulas make the 
formulas easier to learn and recall; on the other hand, the^aj^t j^nal explana- 
tory and integrative material could detract from time and effort ta^t should 
be spent learning the equations. , i / 

From our perspective, it makes little sense to examine total test scores 
as a function of the degree of understanding incorporated into the text used 
for instruction. Which text is better may well depend upon which processes 
are most important in the solution to a particular problem. For example, 
assume that texts which empahsize rote learning of formulas at the cost of 
deemphasizing explanation do provide an advantage in memorizing formulas. 
Then, problems in which categorization and translation are not difficult 
rtay be more "easily solved by students who have less understanding of the 
material. On the other hand, assume that a problem provides no key words 
which immediately signal the problem's category, or contains considerable^— 
irrelevant information which make categorization or translation difficult; * 

a text which has emphasized understanding may provide a definite advantage 
* 

here. - ^ , v , v „ lx . - < * • 

In view of^ these considerations, we have attempted a more molecular 
analysis of test performance than" is usually the case. For each of several 
text conditions, we have examined variations in performance as a function 



of the type of problem* At a rather gross level, we have formula problems, 

which merely presented certain values (e.g. f 'P(A) = .6, P(B) = .4 n ) and 

necessary conditions ("A and B are independent events") , and required the 

subject to set up the correct results (e.g. "What is P(A and B)? u requires 

% .6 x .4, as the answer) ; and story problems^which were considerably less trans- 

parent. Within the set o^ story problems, performance has been related to 

such variables as the presence or absence of key words (e.g. "and" implies 

^the multiplication of probabilities), the presence of irrelevant values, 

and translation requirements (e.g. whether it was necessary to subtract a 

stated probability from one in order to solve). 

Our perspective has also led us to require subjects to report their 

answers as operations on numbers for example*, a correct a'nswer would be 

.4 x .2, not .08.. This approach also provided^more detailed knowledge of 

the types of errors 'made. . . 

< ^ — - — ^ 

To summarize, we view understanding of mathematical material as a function 
of (1) connections of text concepts and formulas to real-world referrents; 
(2) integratiqp °* concepts and formulas within the text; and (3) explaption 
of formulas. This view has provided a basis for constructing three written 
treatments of elementary probability, presumably varying in the degree to 
which they convey understanding. Our view of the proce^^s involved in solving 
problems has led us to use both formula and story problems, and to emphasize 
analyses of error protocols. 

Method 

^ Subjects f \ 

Forty-eight volunteers from an Introductory Psychology course at the 
University of Massachusetts vere randomly assigned to three text conditions. 
None had previous exposure to probability or statistics. / N 

Texts 

There were three texts, each judged to be representative of actual 
chapters on probability in various introductory statistics textbooks. All 



contained the six formulas shown in 



table 1, 



with one numerical example illus- 



trating the application of each formula. All three, texts began with a brief 



i 



Table 1 

Formulas ' Presented in the Three Texts 
(1) P(A or-B.or C) »- P*(A) -KP(B) + P(0) 
(A, B, and C are mutually exclusive) 
v (2) P(A or B), = P(£^ + P(B) - P (A and B) 
(general case) 
/" (3) P(A and B and^ C) = P(A) x P(B) x P(C) 
(A,. B, and C are independent) 
(4) P(A, B, B, B, A) == P'(A) x P (B) x P(B) x P(B) x P (A) 
-(5) P(A-given B) = P(A and B)/P(B) 
- '(6) P(A and *B) = P(A given B) x P(B) 



T~ 

introduction indicating *that tiie text contained alseries of examples of pro- 
bability calculations, and that each example would be followed by^a formula 
which could be used to solve problems of th&f. type/ All three texts then 
contained a section labelled ."What are the chances?" whi^h^ip-troduced the 
concept ^of /probability and its limiting values. This section noted that 
an action such as tossing a coin or observing the day's weather co\ild have 

several possible outcomes,, that a probability value could be assigned to these 

** > 

outcotnea. and *that these values' ranged from zero to one. 

Despite these similarities, the' texts differed in several ^ways . «A high- 
explanatory text defined jjarobability as a relative frequency of events, and 
e&nnples emphasized the relation between answers and codfff^^f the numbers 
ofways that various events could occxir, A standard text treated probability 
as a measure with certain well-defined properties (e.g. limits of one and 
zero) and used familiar real-world examples (e.g. the probability of v rain 
tomorrQw) , but did not explain probability terms of counting processes. 
A low-explanatory text was more abstract than either of the first two; agide 
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from an initial reference to'dice-throwing, examples were stated in terms' 

of such quantities as P(A) and P(B). 

The texts also varied with respect to the degree of linkage among the^ 

equations presented. Only the high-explanatory text showed that one equation 

♦ 

was a special case of another, or that one equation could be derived from 
another. For example, this text pointed out that the probability of the 
union of two mutually exclusive events (Equation 1) was a special case of 
the probability of the ufcion of any two events - (Equation 2). 

Finally, the texts varied in the degree^ of explanation provided fpr 
each equation. In the Cigh-explanatory text, Venn and branch diagrams were 
used to represent the information in the examples, and the equations applied 
in these examples were explained in terms of these diagrams. Neither of 
the other texts provided such explanations. For example, although all three 
texts stated that the probability of the joint occurrence of two independent 
events was the product of their probabilities and provided an example of 
this, only the high-explanatory text attempted to demonstrate why probabilities 
were multiplied in such cases. 

In summary, all texts contained an example followed t by the equation, 



as we 



well as sufficient definit4^ns of terms such as mutual exclusivity and 
r^endence. In the high-explanatory text, the relation of^robability 



to counting was emphasized, pictorial aids were used, and the equation was 
presented as a natural development ensuing from the example. 

\ 

Performance Tests 

Two tests were constructed, each containing two sets of six story problems 
and two sets of six formula problems. Within each set, t^ere\was one problem 

}f slats within 
e four sequences, 

with the constraint that story and formula sets would alternate. Half of 
the sut^ects received one test first and the remaining subjects received 
the other test first. 



testing each o-f the six equations in the text. The order 

A # 

a test was counterbalanced over subjects so that there we 



Formula and story problems differed ir> the degree to which the given 

\ . 

information was explicit in the question,. For example, a formula problem 
^testing knowledge of Equat4<^l would state "POO = A and P(B) = .3. If 
A and B are incompatible, what is the value of P(A or B) ?" ^n^ ^oi^rastv. 
a story problem testing the same/Knowledge would read as follows; "The probability 

" / 



is ,20 that Jimmy will win a raq^ and .30 that he will finish second. What 
is the probability that he will win or finish second?" 
Procedure 

Subjects were tested individually. They were ^nstructed to read the., 
text booklets silently and at their qwn rate, trying to understand the q^plana- 
tions and examples. The standard and low-explanatory groups, whose texts ^ 
, were four pages in length, were told that the y* would have 15 minutes to finish; 
the high-explanatory group, whose text was 14 pages in length,' was given 
25 minutes. These limits, based on pilot subjects, were ^imposed to discourage^ 
any individual from either quickly scanning the texts, or taking an in-~ : 

. r 

ordinately long time. All subjects finished reading their texts in the given 

time period for their group. .Subjects in the low-explanatory and standard, 

groups tended to rate the study time more favorably than subjects in the 

high-explanatory text condition, but this difference was not significant. 

i * i 

At the completion of the study period, subjects were givfen two minutes 

y 

to stu^y a list of 15 words unrelated to probability and were then required 
to write down as many words as ttjey could reca'll on a blank sheet of paper. _ 
Upon completion of the interpolated task, subjects were given/the test. 
There was one problem on a page with space to do any work which the^ subject 
felt might" help in the solution. Subjects were given as much time as they 
required to complete- the test; most required about 30 minutes. Upon completing 
• the problems, subjects were asked to return 48 hours later. On the secopd 
test day, subjects w#fe given the second form of the test. The order of 
presentation of the test forms w/is counterbalanced 




Results and Discussion 
Performance test data were scored according to both a strict and a lenient 
criterion. The strict criterion required that the subject set up the answer 

..Correctly in all respects; the lenient criterion gave credit if the subject 
used the correct equation but, for example, inserted the wrong values. This 
distinction applies only to performances on story problems; strict and lenient 
scores were the same for formula, problems . \ 
A Summary 6f Test Data 

Figure 1 (next page) summarizes the data. The points on the lines are 
for the strict criterion; the disconnected open symbols are for the^lenient 
criterion. -There ctre two important observations to be made "about the means. 
First considering the strict-criterion means, the relative performances 
of the three text conditions- strongly depend upon the type of problem (p < .001) 
The two nonexplanatory groups appear to know the formulas better than the high- 
explanatory group but they have more difficulty in apply ijng that knowledge 
to story problems. ' • 

The second point to note, in Figure 1 is that strict-criterion scores 
are significantly lower than lenient-criterion scores for the two nonexplanatory 
groups (p. < .001) . 'Apparently, subjects in these two groups may correctly 

.categorize the problem arrd recall, the appropriate, fotmula but are not able 

« 0 

to translate the values in the problem statement. In^ contrast, when high- 
explanatory subjects know what operations are to be performed to solve a 
problem, they usually perform them on the correct, set of values; there is 
little difference between their strict and lenient-criterion scores* ' . 
• A Mo'rte- Detailed Analysis of Individual Problems 

"/ tM 'it* is impractical to. examine the numbers and types o-f errors x made to 

each o£..the 24 story problems ( for each of the three text conditions. Therefore, 

i , . - 

we will focus on those problems testing knowledge of Equations 1, 3, and 

, . . - ' ' . / - r . 1 

the ptobletas .within these sets reveal the-widest variation in proportions 



of correct responses. In considering these problems, we ask: Why is' there 
marked variation in proportion correct to problems requiring the same equation 
for solution? And why is this pattern of variation affected by text condition? 

The low-explanatory and standard groups performed in similar fashion 
on all problems. Therefore, in the interests of simplifying the presentation, 
we have treated them as a single condition which we have labeled "nonexplana- 
tory." We will refer to the high-explanatory group as "explanatory 

Equation 1 . Performances on these four problems indicate that subjects 
in the nonexplanatory condition are sensitive to aspects of the problem state- 
ment which have little effect' upon the performance of subjects in the explana- 
tory condition. Proportions of correct responses in the nonexplanatory group 
varied greatly across the four problems" testing knowledge of Equation 1: 
1 (a) .375, (b) .4375, (c) .625, and (d) .7188. In contrast, the performances 
of those in the explanatory group are not only much better but also quite 
stable: (a) .9375, (b) .75, <c) .8125, and (d) .8125. 

• Examining the answers to (a) and (b) more closely Ss^ find clear evidence 
of failures to categorize or translate these problems correctly, but only 
in the nonexplanatory condition. Consider problem (b). 



"In the National Women's Ping Pong League, Los Angeles has 
a .3 chance of winning, San Francisco has a .2 chance, San Diego has 
a .1 chance, New York has a .1 chance^Bfen'd Philadelphia has a .3 chance, 
What is the probability that a west coSt team will win?" 

Note that th^s problem contains irrelevant information; the^ stated pro- 



babilities for New York and Philadelphia are unnecessary. Subjects in the 
nonexplanatory condition attempted to use these probabilities; five subjects 
subtracted .1-+ ,3 from the correct sum (or from the product of the /west 
coast probabilities), four others multiplied all five probabilities, and 
two others divided, by .1 + .3. Only one subject in the explanatory condition 
attempted to incorporate the irrelevant values into his aitwer. Apparently,' 
subjects who do riot understand the rationale for using the equations attempt 
to use any information presented to them. In this example,, the subjects 
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cited misclassif icd the problem, using Equations 2, 4, or 5 ixv an attempt 
to incorporate all the values stated. 

/ 

Problem (b) also indicates that subjects, in the nonexplanatory Condition 
have difficulty when there is PC key word to provide a basis for classifica- 
tion. Eight^e-f the 32 subjects in this condition used Equation 3 rather 
than Equation 1, multiplying probabilities instead of adding them. It is 
revealing that the only other of the four problems in which ^this type of' 
error was frequent ('nine of 12 subjects who made errors) was (c) ; (b)* and 
(c) are the only two problems in this set in which the problem does not speci- 
fically request the probability of somfejfrang or ^something. It is also im- 
portant to note that this misclassif ication error occurred Only once for 
problem (b) and once for^problera (c) in- tfuz explanatory condition ^ 

Even when subjects in the nonexplanatory condition used the correct 
equation, they may have had difficulty translating the story; that is, 
correctly inserting values from the story into the equation. This* was a 
major source of difficulty for the nonexplanatory group in problem (a) . The 
problem presents frequencies of different color marbles in a jar, ancK then 
asks for the probability of drawing a red or white marble. Five subjects 
misclassif ied the problem, attempting to incorporate frequencies of orange 
and blue marbles into their answer. Most of the errors; however, reveal 
a lack of understanding of* the relation between frequency and probability. 
Subjects often failed to divide the sum of the numbers of red and white marbles 
by the total number of marbles, or divided by 100. Note that this problem 
was the piost difficult for the nonexplanatory group but the easiest for the 
v explanatory group who had the relation between frequency and probability 
explained in their text. 

The performance of the nonexplanatory group on problem (d) provides 
further evidence of the validity of our analysis of classification and translation 
errors. This problem evoked the best performance from the nonexplanatory 



group. Not surprisingly, in view of our analysis, it states exactly two 
values, includes the key word "or, 11 and requires no translation of the stated 
values. 

Equation 3 . The explanatory group did not perform as well on these 
problems as on those testing knowledge of Equation 1. We suspect that the 
rationale for multiplication of probabilities is more difficult to understand 
than that for adding probabilities , 

In one respect, performances oti this problem set were similar to those 
on the first problem set: proportion correct was fairly stable over problems 
for the explanatory group and quite variable for the nonexplanatory group • 
For the explanatory group, these proportions were (a) .5, (b) ,5625, (c) ,625, 
and (d) ,5625; for the nonexplanatory group, the proportions were (a) ,2813, 
(b) ,5, (c) ,625, and (d) ,8438, As with the first problem set, it appears 
that subjects in the nonexplanatory condition are more sensitive to the statement 
of the- problem than- arersubjects in the explanatory condition. 

Much of the variability exhibited by the nonexplanatory group can again 
be traced to variations in the problems with respect to irrelevant information 
and translation requirements. Problem (b) presented values that were redundant 
(in essence, presenting P(A), P(not-A), P(B), P(not-B), Four subjects in** 
the nonexplanatory group used the wrong equation in an attempt to incorporate 
all four values in their answer (e.g, ",2 x .4 x .6 x ,8 !t ) and one made what 
we would characterize as ^ translation error, multiplying ,2 by ,4. In con- 
trast, no subject in the explanatory group used numbers other than .2 and 
• 6 in their answers. 

The answers to problem (a) indicate a serious translation difficulty. 
The problem states P(A) and P(not-B), and asks for P(A and B) , Of 23 subjects 
in the nonexplanatory coriditi6n who erred in response to this problem, 11 
multiplied P(A) by P(B); only one subject in the explanatory condition did 
this; 



In contrast to performances <?n problem (a), the nonexplanatory group 

did very well in answering problem (d) . This states: 

".7 of the members of the state legislature favor a 
particular bill, .6 of the members are Democrats. If party and vote 
are independent, what is the probability that the next legislator you 
meet is a Democrat and votes for the bill? 11 

We suggest that this problem was very easy for the nonexplanatory group 
because they have memorized the rule: "Whenever the probabilities of something 
and something, else are required, multiply their individual probabilities. 11 
In problem (d) , the* word* "and 11 is present and there are only two possible 
values to multiply. Fortunately, they are the right two values. The same 
circumstances hold for problem (a) except that the values given are not right; 
one value must b& subtracted from 1.0 before multiplying. 

Equation 4 . This equation is the general case of Equation 3 and, there- 
fore, it would seem that subjects should score about the same on this problem 
set as on the earlier one. That was the case for subjects in the explanatory 
group; in the nonexplanatory group, however, the average proportion correct 
was .22 lower for the problems testing knqwledge of Equation A than for those 
testing knowledge of Equation 3. For the explanatory condition, the proportions 
correct were, as in the other problem sets, relatively stable: (a) .5, 
(b) .4375, (c) .625,. and (d) .5625. Also, as before, the nonexplanatory A 
group exhibits greater variability of performance over the four problems: 
the relevant proportions were (a) ..1563, (b) .1875, (c) .3438, and (d) .625. 

That the nonexplanatory group had" greater difficulty with these J:our 
problems than with the third problem set" may reflect two factors. First, 
these problems typically omit explicit use of the key word "and; 11 this may 
cause classification problems. Second^ the probability of the complementary 
event is often not expis^cifly stated; thus, there may be translation diffi- 
culties- \ ^ 

Both conditions exhibit about 16% classification errors as indicated 
by use of operations other than multiplication. More striking,' at least 



in the nonexplanatory condition, were errors of translation indicated by 
using the stated probability when its complement was appropriate. Averaging 
over problems (a), (b) , and (c) , 11.67 subjects in the nonexplanatory group 
made such errors. Only 2.33 subjects did so in the explanatory group. 

The nonexplanatory group performed reasonably well on problem (d) which 
states : 

"A hockey team wins with probability .5, loses with probability 
of .3, and ties with probability of .2. Vttiat is the probability that 
in the next four games, the team wins, then loses, then wins, then loses?" 

The statement of this problem more closely parallels the statement of Equation k 

than does that of the other problems in this set, thus reducing the likelihood 
m 

of misclassif ication. Furthermore, all the required probabilities are stated; 
translation difficulties are presumably minimal. 

Conclusions 

Very different patterns of knowledge appear to be present in subjects 
in the* nonexplanatory (standard and low-explanatory texts) and explanatory 
conditions. Subjects in the two nonexplanatory groups performed considerably 
less well on story than on formula problems, and often used the correct formula 
ior a problem (that is, met the lenient criterion) but failed to solve it. 
A closer look at answers to story problems revealed that- subjects in the 
nonexplanatory conditons often required the explicit presence of key words 
which unambiguously pointed to certain operations, tended to misclassify 
problems in the presence of irrelevant or redundant information, and made 
many errors when the values in the, story required modification before insertion 
into the formula* In contrast, subjects in the high-explanatory condition 
performed equally well on story and formula problems, tended to solve whenever . 
they showed evidence of knowing the appropriate formula, and were considerably 
less hindered by absence of key words, the presence of irrelevant information, 
and the need to translate values in the story. The situation may be summariEed 
by noting that, although all oiir subjects were novices, those in the high- 
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explanatory condition seem to share more of the characteristics of experts 

j 

than do those in the other two conditions. 

The results of more recent studies lead us to believe that the solution 
processes in the explanatory and nonexplanatory conditions are qualitatively 
different. In one study, subjects in a high-explanatory condition had only 
607 correct recall of equations (as opposed to 90% for a standard group), 
but were able to perform correctly on 53% of story problems. This result 
suggests to us (as does the equivalence of formula and story scores in the 
study just detailed) that subjects in the high-explanatory condition often 
may not retrieve the formula at all. Another study in which subjects thought 
aloud while solving supports this conjecture; after studying a high-explanatory 
text, students tended to attempt to construct solutions, ojzten using the 
diagrammatic aids provided by the text. Rather than recalling formulas, 
they attempted to recall examples frcn^he text which they felt were similar 
to the problem at' hand, arfd to^use these as a model for solution. 

Much remains to be done. High on the agenda must be the use of other 
measures of comprehension — both problems demanding more complex processing 
and transfer to new materials such as the Binomial and Bayes 1 theorems. We 

suspect that such research will provide further evidence of the role under- 

t 

standing can play in mathematical problem-solving. 
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